Fix block upload worker leaks after an error - #8
Open
ClaudiuSchuster wants to merge 2 commits into
Open
Conversation
Receive every block upload result before returning the first error so all workers can release their semaphore slots. Buffer the result channel and return immediately when slot acquisition fails. Add a regression test which repeats failing batches and then acquires the full semaphore capacity.
ClaudiuSchuster
force-pushed
the
fix/drain-upload-block-workers
branch
from
August 27, 2026 19:28
a386958 to
1d2e00d
Compare
This was referenced Aug 28, 2026
Retry only failed encrypted blocks with fresh upload links and bounded context-aware backoff. Preserve successful blocks and return terminal or exhausted errors without replaying the complete file stream. Refs oss-singularity/proton-drive-linux#42
Author
|
Additional real-world validation from the downstream PDrive integration:
This supports the semaphore-release behavior under repeated real backend failures, beyond the unit and race tests already listed in the PR. A separate bounded retry for only the failed encrypted blocks is tracked in oss-singularity/proton-drive-linux#42 and was validated in oss-singularity/Proton-API-Bridge#2. I am deliberately keeping that follow-up out of this PR so this worker-lifecycle fix remains focused. Once this prerequisite lands, the retry can be proposed upstream as a single independent commit. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
uploadPendingBlocksreturned as soon as it received the first block-upload error. The remaining goroutines could then block while sending to the unbuffered result channel, still holding their weighted-semaphore slots. Repeated retries progressively consumed all block-upload slots and left later uploads unable to start.The acquisition-error path also continued to a deferred
Releaseeven though no slot had been acquired.Fix
Verification
Validated with Go 1.26 and golangci-lint 2.9.0, matching the repository workflow:
All commands pass. The repository's credential-gated Proton integration tests are skipped by their existing guard; all unit tests, including the new regression test, run successfully.
The fork's repository workflow also passes on this exact commit: Lint and Test.